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Abstract 

This letter discusses the problem of testing the degree of randomness within an image, particularly for 
a shuffled or encrypted image. Its key contributions are: 1) a mathematical model of perfectly shuffled 
images; 2) the derivation of the theoretical distribution of pixel differences; 3) a new Z-test based 
approach to differentiate whether or not a test image is perfectly shuffled; and 4) a randomized algorithm 
to unbiasedly evaluate the degree of randomness within a given image. Simulation results show that the 
proposed method is robust and effective in evaluating the degree of randomness within an image, and 
may often be more suitable for image applications than commonly used testing schemes designed for 
binary data like NIST 800-22. The developed method may be also useful as a first step in determining 
whether or not a shuffling or encryption scheme is suitable for a particular cryptographic application. 

I. Introduction 

Currently digital images are a major information resource in real life. A large amount of digital images 
are private, sensitive, or even classified. Much research has been done to ensure security during image 
transmission or storage. Two typical protection methods are image shuffling (IS) [ 1 ] and image encryption 
(IE) 0, 0, where IS only scrambles pixel positions, while IE changes both pixel positions and values. 
Regardless of the technical details of a protection method, a key question that need to be answered is 
'Is this method secure?'. Unfortunately, this raises a dilemma: if a method is secure, then it must be 
invulnerable to all attacks; however, it is impossible to list all existing but unknown attacks, or those will 
be developed in the future. Thus, all we can do is to test whether it is secure under our known attacks. 
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Among various attacks, one family is statistical attacks [4], which take advantage of leaked infor- 
mation from poorly protected images. This leaked information always exists, unless a protected image 
is indistinguishable from a random one. Therefore, the indistinguishability from a random image is a 
necessary (but not sufficient) condition for a secure protection method, and evaluating the degree of 
randomness becomes a prerequisite. For example, FIPS 140-2 tests Q and NIST 800-22 test suite @ 
are suggested for cryptographic applications. However, they are merely designed for one-dimensional 
(ID) data encryption (DE), quite different from two-dimensional (2D) IS or IE. For instance, DE expects 
random 0s and Is that are independent identically distributed (i.i.d.) with equal probabilities. IS leads to 
a pixel distribution identical to that of the original image, and thus can be arbitrary. IE should lead to a 
uniform distribution, but it is unclear that how to tailor pixels from a 2D image into a ID bit sequence, 
so that DE evaluation tools can be properly applied. Thus, additional randomness evaluation tools for 
images need to be developed. 

Although some evaluation methods (EM) have been presented, the following four serious challenges 
still should be considered (summarized from [7]): 1) pixel values and positions should be both considered; 
2) the randomness degree should effectively reflect the relationship between a shuffled image and the 
corresponding method; 3) the evaluation should be dependent only on a test image; and 4) the method 
complexity should be independent of a test image. However, so far an effective approach does not exist [8]. 
Existing EMs for IS and IE can be classified into two groups: subjective methods and objective methods. 
The former group commonly uses empirical knowledge to quantify the randomness of a shuffled or 
encrypted image (3, Q, iPTOll . while the latter group tests the image randomness with respect to derived 
statistics ifTTTl . lfT2l . 1131 . The use of pixel differences for image randomness has been explored for 
a long time Q, ifTOl . but these heuristic, subjective methods allow only quantitative evaluations. 
Nevertheless, all these mentioned methods are either for IS or IE, but not both of them. 

In this letter, we propose a new quality EM for both IS and IE using pixel differences, which solves 
all four mentioned challenges. The rest of the letter is organized as: Sec. II studies the perfectly shuffled 
image model; Sec. Ill constructs hypothesis tests for our randomness EM; Sec. IV discusses a special 
case for IE; Sec. V provides simulation results; and a summary of the paper with conclusions is given 
in Sec. VI. 
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II. Perfectly Shuffled Images 



A. What Is a Perfectly Shuffled Image? 

Before developing a randomness EM, one has to answer the question 'what is a perfectly shuffled 
image?'. Altough there are other answers, we believe it is the one with the properties that 1) all neighbor 
pixels are independent and 2) only random-like patterns exist within the image. This means that pixels 
in a perfectly shuffled image are i.i.d. just as a random image. Thus, a perfectly shuffled image can be 
defined as: 

Definition 1: (A Perfectly Shuffled Image) An L-intensity scale image ~K = {xi}i e n on a 2D domain O 
with a pixel distribution V=[po,pi, • • - ,pl~i] fSfco Pk = 1) is called perfectly shuffled, if each pixel is 
i.i.d. with distribution V, where p k is the probability of seeing a pixel at the k til scale in X and <5(.) is 
the Kronecker delta function with 1 at and elsewhere. 



B. Theoretical Statistics 

Theorem 1. If image X is perfectly shuffled, then the pixel difference pi k =\ii— Xk\ between two arbitrary 
distinctive pixels x\ and x k with Z, fee and l^k satisfy the distribution 



(1) 



P d = Pr{pik = d)={ 



EkZoPlXd = 
2 2~2k=d Pk-dPk , if d e [1 , L- 1] 



(2) 



Proof: Firstly, compute the probability for d = 



Pr(flk = 0) = Pr(x ; = x k ) 



= Lfc=0 PkPk 




Secondly, for arbitrary difference d / 0, we have 



Pr(m = d) = Pr(x; = x k + d) + Pr(x ; = x k - d) 



= E%=oM x i = x k+d\x k )Pv(x k ) 



+ 2~2k=d PT ( x i = x k-d\x k )Pr(x k ) 

= LfcO Pk+dPk + Lfcrf Pk-dPk 
= 2 2~2k^Pk-dPk 
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Finally, it is noticeable that 

Ed=o Mpik = d) =Efc=o pI + 2 Efci 1 Efc] p*-dP* 
=(Eto^) 2 = i 2 = i 

is indeed a discrete probability distribution. ■ 
Remark. 1) The distribution ([2]) is image dependent, because V is image dependent. 2) P can be obtained 
from a shuffled image directly, because V doesnot change before and after IS. 

Theorem 2. Let p~m~ = Y^ILi Pum/ 171 be the average of the differences for m pairs pixels with m>30 
and U™ x li n U™ 1 /c i = 0. Then ^ follows the Normal distribution 

M{p,(J 2 ' /m) (3) 

with the mean and variance as 

P = Ed=o dp d ^d a 2 = Ed=o ^ " M 2 - (4) 
Proof: Straight-forward by using the central limit theorem for i.i.d. pi^fi following distribution (|2]). ■ 

III. Evaluating Image Shuffling Randomness using Pixel Differences 
A. Hypothesis Tests for Perfectly Shuffled Images 

Knowing the theoretical distribution of p^, we are able to construct hypothesis tests for IS. In particular, 
we can test image randomness by comparing the theoretical average with the sample average pixel 
difference of m pairs pixels as follows: 

Hq: Null hypothesis that image Y is indistinguishable from a perfectly shuffled image X, i.e. p~m = 

Pm- 

Hi : Alternative hypothesis that p~m ^ p~^. 

where = £™i \yu ~ IfcJ/m and Uf =1 k n U^h = 0. 

Since p~^^J\f{p,a 2 /m), this hypothesis test is a Z-test, where the test statistic can be constructed as 

z= P™-P mdz ^ Af , 01 s (5) 
a/y/m 

Therefore, given a significance level a, we can compute the critical values under the null hypothesis as 



I 1 + *a/2°7 V^" and Prn* + = P~ $ ' a j 2 ° '/ ' V™ ( 6 ) 
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where is the inverse cumulative distribution function of the standard Gaussian A/"(0, 1). If the sample 
average ^ [(hn*~ , pin* + ], we reject the null hypothesis and claim Y is distinguishable from a perfectly 
shuffled image. 



B. A New Image Shuffling Quality Evaluation Method 

It is noticeable that the proposed hypothesis tests are valid for m pairs of pixels regardless of their 
spatial configuration, e.g. horizontal neighbor pixels, adjacent image blocks etc. In practice, however, 
we have to unbiasedly test pixels with all possible spatial configurations. Otherwise, we cannot fairly 
quantify the degree of randomness. One solution is to use the randomized algorithm 1. 



Algorithm 1 (a,N,m,T) Pixel Difference Test for IS 

Require: Y is a L intensity scale image and a is the significance level 
Require: N=# of tests, m =# pixel pairs and T= # of evaluation times 
Ensure: r is the success rate for pixels in Y passing the hypothesis test. 

1: Find sample intensity distribution T' = {Pk}ke[o,L—i] f° r image Y using {TJ 

2: Compute the pixel difference distribution Pr(p) = {Pd}d£[0,L—l] usm g Q 

3: Compute the mean fi and variance <r 2 using J4} 

4: Compute critical values pm*~ an d Pm* + using |6j 

5: for t = 1 to T do 

6: Initialize rt = 

7: for i = 1 to JV do 

8: Pull m pairs of image pixels with a random configuration w\o repetitions 

YbH.Vh>JJ ' 'Vim} and Yh k={Vk 1 ,- ■ ■ ,Vk m }- 

9: Compute p m = J2i=l \Vk ~ Vk t \/m 

10: if fhn 6 [fhn*~ ,fhn* + ] then 

11: r t ++ 
12: end if 

13: end for 
14: end for 

15: r = max te[1>T ]{ri}/iV. 



Algorithm 1 considers both pixel values and positions, reflects the randomness within a test image, 
relies only on a test image and has a constant complexity O(NTm). Therefore, it successfully solves all 
four challenges specified in 0. 

Assume Y is perfectly shuffled. The four parameters in Algorithm 1 have the following impacts. 
Parameter a is close related to the pass times r t , whose mean and variance are 

E[r t ] = (l-a)N&ndElrt-E[r t ]) 2 \ = Na(L-a). (7) 

Parameter influences the number of spatial configurations and also the number of pixels used in the 
evaluation. Thus, a larger implies a more fair evaluation. 

Parameter m has two related effects: 1) the larger m is, the smaller the variance a 2 /m is in ([3]); 2) 
the larger the m is, the poorer the localization of the test. The former effect prefers a large m to attain 
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a small variance, while the latter one prefers a small m to test a local pixel configuration. Therefore, 
an optimal m* should be the one balancing both effects. To do so, we define the loss function ([8]) for 
m, where term a 2 /m reflects the loss in the test accuracy, term m 2 /|f2| reflects the loss in localization 
capacity, and A is the coefficient indicating the relative importance of the two types of loss. 

fy(m) = a 2 /m + \m 2 /\Q\ (8) 

Hence, the optimal m* minimizes the loss function \&(m) 

m* = argmin m6[1) | n |] *(m) (9) 



and is of the form (flO]), where [•] is the integer rounding function towards infinity. 



m* = \^/a 2 \n\/(2X)] (10) 

Finally, parameter T plays an important role to reduce the type I error (false positive error) of the 
evaluation. As T— too, the type I error goes to zero. 

lim Pr(r<l-a|# )= lim Pr(max te[1)T] {r t } < E[r t ]\H ) =H^°Pi(r t <E[r t ]\H o ) = (11) 

T— >oo T— too 

IV. Special Case for Image Encryption 

Because ideally an encrypted image is random-like with a uniform pixel distribution ifTTl . lfl2l . the 
derived statistics, hypothesis tests, and the proposed EM are also applicable to IE, by simply using 
V=U[0, L-l] (i.e. p k = 1/L in Q). 

Specifically, the pixel difference distribution for perfectly encrypted images follows the triangular 
distribution ( [12] ) and the corresponding average pixel difference for m pair pixels p e m follows the Normal 
distribution / o,^~AA(^ e , al/m). 

{ 1/L,ifd = 

Pr e ( Plk = d) = { (12) 
[2(L-d)/L 2 , ifde[l,L-l] 

Me = (L 2 - 1) /(3L) and a\ = (L 2 - 1) (L 2 + 2) /(18L 2 ) (13) 

Table |l] gives the theoretical numerical results of fi e and a e under various image types, where L =2, 
256 and 65536 refer to the binary image, 8-bit grayscale image and 16-bit grayscale image, respectively. 
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The critical values of the hypothesis tests can be computed as 

where m* is the optimal number of independent pixel pairs used in the quality measure and can be 
computed via ( flO] ). 



TABLE I: Pixel Differences Statistics in Perfectly Encrypted Images 





Imaj 


;e Intensity Levels L 


Stat. 


2 


256 65536 


Me 


0.500 


85.332 21845.333 




0.500 


60.340 15446.983 



V. Simulation Results 

We apply the proposed randomness EM (Algorithm 1) for IS and IE to real data. All following 
experiments are done under the MATLAB r2012a environment. Test images are square size 8-bit grayscale 
(i.e. L-256) from the USC-SIPI miscellaneous dataset (detail descriptions are available online l ). 

The parameters N, a and T in Algorithm 1 are set to 1000, 0.05, and 10 respectively. Parameter m is 
optimized with respect to each test image with A = fi/L for IS and A = fi e /L for IE. Related statistics 
for performing the quality evaluation are given in Table |nj where the critical values are computed with 
respect to a-.05. 

Under these parameter settings, it is noticeable that 

Pr(r t < E[r t ]|flo) = E2S ( 10 s °).95 s .05 100 ^ = .4625 

and thus 

Pr(r<l-a\H ) = .4625 10 < 5/10000 

indicating a very small type I error of the evaluation method. 

Tested IS methods are the random permutation method (RPM) HI, row and column shuffling (RCS) 
lfl4l . Arnold transform method (ATM) Q, and Sudoku shuffling (SS) HH. Meanwhile, we also use the 
IE methods including the logistic map encryption (LME) |[T6l , the AES cipher [3] with the electronic 
codebook (ECB) mode and the cipher-block chaining (CBC) mode, respectively. It is worthy to point 
out that the RPM and the AES-CBC are classic methods for IS and IE, respectively. Therefore, it is not 



is available under the page 



http://sipi.usc.edu/database/database.php?volume=misc as of 10/22/2012. 
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TABLE II: Statistics of Pixel Differences for Test Images 



Image Info. 
Name Side 



Stat, for IS 



Stat, for IE 



■ *± 



5.1.09 
5.1.10 
5.1.11 
5.1.12 
5.1.13 
5.1.14 
5.2.08 
5.2.09 
5.2.10 
5.3.01 1024 
5.3.02 1024 
7.1.01 512 
7.1.02 
7.1.03 
7.1.04 
7.1.05 
7.1.06 
7.1.07 
7.1.08 
7.1.09 
7.1.10 
7.2.01 1024 
boat.512 512 
elaine.512 
gray21.512 
numbers.512 
ruler.512 



256 
256 
256 
256 
256 
256 
512 
512 
512 



512 
512 
512 
512 
512 
512 
512 
512 
512 



512 
512 
512 
512 



testpat.lk 1024 



29.58 
51.54 
33.82 
58.51 
51.24 
47.60 
43.75 
40.58 
61.91 
66.54 
37.88 
29.48 
17.00 
27.98 
36.38 
39.71 
37.93 
26.19 
21.85 
40.67 
29.12 
21.27 
49.85 
52.65 
88.72 
70.28 
49.94 
85.28 



25.76 
38.15 
32.31 
55.96 
94.08 
36.37 
35.99 
38.27 
46.46 
47.79 
32.88 
24.41 
26.26 
26.10 
33.02 
29.46 
27.97 
21.92 
26.18 
30.50 
24.38 
29.33 
43.27 
38.35 
62.95 
51.91 
101.20 
64.98 



574 27.47- 
61948.53- 
63831.31- 
766 54.54- 
113245.76- 
61644.73- 
99841.52- 
106638.29- 
1054 59.11- 
1664 64.24- 
1565 36.25- 
879 27.87- 
110915.46- 
935 26.31- 
1002 34.34- 
902 37.79- 
885 36.09- 
85124.72- 
1018 20.24- 
91638.70- 
882 27.51- 
1758 19.89- 
1081 47.27- 
97950.25- 
1145 85.07- 
1088 67.19- 
190245.40- 
1881 82.35- 



31.69 
^54.54 
-36.33 
'62.47 
-56.72 
■50.47 
■45.98 
42.88 
■64.72 
-68.84 
'39.51 
-31.10 
-18.55 
29.65 
38.43 
■41.64 
'39.77 
■27.66 
■23.45 
42.65 
■30.73 
■22.64 
-52.43 
55.05 
92.36 
73.36 
■54.49 
■88.22 



85.33 60 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60. 
85.33 60 



711 80.90 
71180.90 
71180.90 
71180.90 
71180.90 
711 80.90 
341128 81.8 
341128 81.8 
341128 81.8 
341790 82.54. 
341790 82.54 
341128 81.8 
341128 81.8 
341128 81.8 
341128 81.8 
341128 81.8 
341128 81.8 
341128 81.8 
341128 81.8 
341128 81.8 
341128 81.8 
341790 82.54 
341128 81.8 
341128 81.8 
341128 81.8 
341128 81.8 
341128 81.8 
341790 82.54. 



-89.77 
-89.77 
-89.77 
-89.77 
-89.77 
-89.77 
-88.85 
-88.85 
-88.85 
-88.13 
-88.13 
-88.85 
-88.85 
-88.85 
-88.85 
-88.85 
-88.85 
-88.85 
-88.85 
-88.85 
-88.85 
-88.13 
-88.85 
-88.85 
-88.85 
-88.85 
-88.85 
-88.13 



surprising to see good performance scores for these two methods. In contrast, the RCS and ATM are 
known to have mesh-like and slant patterns, respectively. Thus we shall expect to see low scores for these 
two methods. The AES-ECB method is known to be insecure for patterned images, because it cannot 
encrypt two identical blocks to different random patterns with the same key. 

Evaluation results for these methods are listed in Table III. As one can see, RPM and AES-CBC 
have their evaluation scores all above \-a-.95. In contrast, none of evaluation scores for the RCS and 
ATM is above .95 (emphasized by underlines), indicating that one can distinguish their shuffled images 
from those perfectly shuffled. The scores for these two methods vary greatly for different test images, 
especially for the ATM, whose IS performance is known to be related to the period of a random Arnold 
transform. The AES-EBC has good scores for most images but bad scores for highly structured images 
like ruler.512 and testpat.lk. The SS and LME both have good evaluation scores above .95 indicating that 
their shuffled or encrypted images are indistinguishable from random-like. Fig. [T] gives sample results 
of these methods. Once again, the indistinguishability of shuffled or encrypted images from random-like 
ones are only a necessary condition for a method to be secure. All these results are sufficient to conclude 
the methods RCS, ATM and ECB are insecure. 
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TABLE III: Quality Evaluation Scores (1CT 3 ) for Tested Methods 





Original 




IS 






IE 




Test Image 




RPM RCS ATM 


SS 


CBC 


ECB LME 


5.1.09 


479 


960 


904 


489 


959 


960 


964 


959 


5.1.10 


894 


957 


937 


918 


959 


961 


960 


960 


5.1.11 


223 


960 


835 


210 


960 


957 


957 


956 


5.1.12 


271 


961 


836 


272 


962 


957 


961 


956 


5.1.13 


871 


961 


898 


908 


957 


960 


489 


955 


5.1.14 


695 


960 


905 


720 


960 


958 


955 


959 


5.2.08 


668 


958 


836 


673 


960 


964 


964 


965 


5.2.09 


877 


960 


952 


928 


959 


965 


952 


956 


5.2.10 


340 


955 


889 


526 


962 


964 


963 


950 


5.3.01 


387 


957 


937 


418 


958 


960 


962 


957 


5.3.02 


603 


958 


943 


649 


961 


959 


957 


962 


7.1.01 


472 


958 


934 


503 


958 


961 


965 


961 


7.1.02 


354 


968 


936 


354 


965 


961 


963 


966 


7.1.03 


433 


957 


910 


470 


966 


957 


961 


964 


7.1.04 


460 


960 


915 


509 


958 


964 


964 


964 


7.1.05 


759 


959 


920 


789 


957 


956 


962 


959 


7.1.06 


681 


961 


937 


715 


961 


959 


958 


956 


7.1.07 


804 


955 


939 


844 


961 


971 


969 


969 


7.1.08 


560 


959 


896 


525 


964 


956 


951 


966 


7.1.09 


116 


963 


864 


113 


961 


960 


956 


961 


7.1.10 


650 


957 


913 


662 


959 


962 


960 


961 


7.2.01 


598 


958 


929 


668 


963 


961 


954 


953 


boat.512 


302 


957 


843 


319 


958 


956 


960 


957 


elaine.512 


499 


963 


936 


594 


961 


959 


956 


956 


gray21.512 


62 


956 


852 


71 


955 


960 


567 


970 


numbers.512 


707 


961 


916 


777 


958 


965 


967 


962 


ruler.512 


531 


962 


801 


701 


958 


959 


299 


961 


testpat.lk 


222 


957 


894 


269 


959 


957 


628 


956 


Mean 


518.5 


959.2 900.3 556.9 960.0 


960.3 893.7 959.9 


Standard Deviation 


229.8 


2.8 


40.9 


239.5 


2.5 


3.4 


172.2 


4.8 



VI. Conclusion 

We introduced a new randomness (quality) evaluation method for IS. It tests whether an image is 
indistinguishable from a perfectly shuffled one, whose image pixels are i.i.d. everywhere. The proposed 
method can be used as a prerequisite test to claim an IS or IE method is secure. Simulation results 
indicate that this method is accurate and effective for quality evaluation for IS and IE methods. Moreover, 
it can be used as an indicator of quality control. That is, if a shuffled or encrypted image fails to 
reach the expected evaluation score, it should be re-shuffled or encrypted using another key. Further, the 
developed method may be useful as a first step in determining whether or not an IE scheme is suitable 
for a particular cryptographic application. The proposed method is of a constant complexity and can be 
easily implemented. Thus, it is well suited for practical applications. A MATLAB implementation can 
be provided on request. 
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(gl) numbers.ECB 



(a2) ruler.RPM 



(b2) ruler.RCS 



(c2) ruler.ATM 
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(e2) Original image: ruler.512 



(f2) ruler.CBC (g2) ruler.ECB 

Fig. 1: Shuffled and encrypted images 



(dl) numbers. SS 



(hi) numbers.LME 



(d2) ruler.SS 



(h2) ruler.LME 
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